Smart Gradient - An adaptive technique for improving gradient estimation

Computing the gradient of a function provides fundamental information about its behavior. This is essential for several applications and algorithms across various fields. One common application that require gradients are optimization techniques such as stochastic descent, Newton's method trust region methods. However, these methods usually requires numerical computation at every iteration which prone to errors. We propose simple limited-memory technique improving accuracy numerically computed in this gradient-based framework by exploiting (1) coordinate transformation (2) history previously taken descent directions. The verified empirically extensive experimentation on both test functions real data applications. proposed implemented R package smartGrad C++.